
import pandas as pd
from fast_ml.tools import EDA


df = pd.read_csv('house_prices.csv')
df.head()


eda_report = EDA.generate_report(df, target='SalePrice', model_type='reg')
eda_report.report_title_ = 'EDA Report for House Price Dataset (Regression)'
eda_report.report_user_ = 'Samarth Agrawal (using Fast ML)'
eda_report.show()

(1460, 81)

Number of Numerical Variables detected :  38
Number of Categorical Variables detected :  43
Number of DateTime Variables detected :  0

After calibration number of Variables used for Numerical EDA :  20
After calibration number of Variables used for Categorical EDA :  61
Out of 61 Categorical EDA Variables 0 have cardinality of more than 200: []
After calibration number of Variables used for DateTime EDA :  0

924.03 KB

Number of Missing Values: 0
Percentage : 0.0 %

Number of Missing Values: 259
Percentage : 18.0 %

Number of Missing Values: 0
Percentage : 0.0 %

Number of Missing Values: 0
Percentage : 0.0 %

Number of Missing Values: 0
Percentage : 0.0 %

Number of Missing Values: 8
Percentage : 1.0 %

Number of Missing Values: 0
Percentage : 0.0 %

	Id	MSSubClass	MSZoning	LotFrontage	LotArea	Street	Alley	LotShape	LandContour	Utilities	...	PoolQC	Fence	MiscFeature	MoSold	YrSold	SaleType	SaleCondition	SalePrice
0	1	60	RL	65.0	8450	Pave	NaN	Reg	Lvl	AllPub	...	NaN	NaN	NaN	2	2008	WD	Normal	208500
1	2	20	RL	80.0	9600	Pave	NaN	Reg	Lvl	AllPub	...	NaN	NaN	NaN	5	2007	WD	Normal	181500
2	3	60	RL	68.0	11250	Pave	NaN	IR1	Lvl	AllPub	...	NaN	NaN	NaN	9	2008	WD	Normal	223500
3	4	70	RL	60.0	9550	Pave	NaN	IR1	Lvl	AllPub	...	NaN	NaN	NaN	2	2006	WD	Abnorml	140000
4	5	60	RL	84.0	14260	Pave	NaN	IR1	Lvl	AllPub	...	NaN	NaN	NaN	12	2008	WD	Normal	250000

	data_type	data_type_grp	num_unique_values	sample_unique_values	num_missing	perc_missing
Id	int64	Numerical	1460	[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]	0	0
MSSubClass	int64	Numerical	15	[60, 20, 70, 50, 190, 45, 90, 120, 30, 85]	0	0
MSZoning	object	Categorical	5	[RL, RM, C (all), FV, RH]	0	0
LotFrontage	float64	Numerical	110	[65.0, 80.0, 68.0, 60.0, 84.0, 85.0, 75.0, nan...	259	17.7397
LotArea	int64	Numerical	1073	[8450, 9600, 11250, 9550, 14260, 14115, 10084,...	0	0
Street	object	Categorical	2	[Pave, Grvl]	0	0
Alley	object	Categorical	2	[nan, Grvl, Pave]	1369	93.7671
LotShape	object	Categorical	4	[Reg, IR1, IR2, IR3]	0	0
LandContour	object	Categorical	4	[Lvl, Bnk, Low, HLS]	0	0
Utilities	object	Categorical	2	[AllPub, NoSeWa]	0	0
LotConfig	object	Categorical	5	[Inside, FR2, Corner, CulDSac, FR3]	0	0
LandSlope	object	Categorical	3	[Gtl, Mod, Sev]	0	0
Neighborhood	object	Categorical	25	[CollgCr, Veenker, Crawfor, NoRidge, Mitchel, ...	0	0
Condition1	object	Categorical	9	[Norm, Feedr, PosN, Artery, RRAe, RRNn, RRAn, ...	0	0
Condition2	object	Categorical	8	[Norm, Artery, RRNn, Feedr, PosN, PosA, RRAn, ...	0	0
BldgType	object	Categorical	5	[1Fam, 2fmCon, Duplex, TwnhsE, Twnhs]	0	0
HouseStyle	object	Categorical	8	[2Story, 1Story, 1.5Fin, 1.5Unf, SFoyer, SLvl,...	0	0
OverallQual	int64	Numerical	10	[7, 6, 8, 5, 9, 4, 10, 3, 1, 2]	0	0
OverallCond	int64	Numerical	9	[5, 8, 6, 7, 4, 2, 3, 9, 1]	0	0
YearBuilt	int64	Numerical	112	[2003, 1976, 2001, 1915, 2000, 1993, 2004, 197...	0	0
YearRemodAdd	int64	Numerical	61	[2003, 1976, 2002, 1970, 2000, 1995, 2005, 197...	0	0
RoofStyle	object	Categorical	6	[Gable, Hip, Gambrel, Mansard, Flat, Shed]	0	0
RoofMatl	object	Categorical	8	[CompShg, WdShngl, Metal, WdShake, Membran, Ta...	0	0
Exterior1st	object	Categorical	15	[VinylSd, MetalSd, Wd Sdng, HdBoard, BrkFace, ...	0	0
Exterior2nd	object	Categorical	16	[VinylSd, MetalSd, Wd Shng, HdBoard, Plywood, ...	0	0
MasVnrType	object	Categorical	4	[BrkFace, None, Stone, BrkCmn, nan]	8	0.547945
MasVnrArea	float64	Numerical	327	[196.0, 0.0, 162.0, 350.0, 186.0, 240.0, 286.0...	8	0.547945
ExterQual	object	Categorical	4	[Gd, TA, Ex, Fa]	0	0
ExterCond	object	Categorical	5	[TA, Gd, Fa, Po, Ex]	0	0
Foundation	object	Categorical	6	[PConc, CBlock, BrkTil, Wood, Slab, Stone]	0	0
BsmtQual	object	Categorical	4	[Gd, TA, Ex, nan, Fa]	37	2.53425
BsmtCond	object	Categorical	4	[TA, Gd, nan, Fa, Po]	37	2.53425
BsmtExposure	object	Categorical	4	[No, Gd, Mn, Av, nan]	38	2.60274
BsmtFinType1	object	Categorical	6	[GLQ, ALQ, Unf, Rec, BLQ, nan, LwQ]	37	2.53425
BsmtFinSF1	int64	Numerical	637	[706, 978, 486, 216, 655, 732, 1369, 859, 0, 851]	0	0
BsmtFinType2	object	Categorical	6	[Unf, BLQ, nan, ALQ, Rec, LwQ, GLQ]	38	2.60274
BsmtFinSF2	int64	Numerical	144	[0, 32, 668, 486, 93, 491, 506, 712, 362, 41]	0	0
BsmtUnfSF	int64	Numerical	780	[150, 284, 434, 540, 490, 64, 317, 216, 952, 140]	0	0
TotalBsmtSF	int64	Numerical	721	[856, 1262, 920, 756, 1145, 796, 1686, 1107, 9...	0	0
Heating	object	Categorical	6	[GasA, GasW, Grav, Wall, OthW, Floor]	0	0
HeatingQC	object	Categorical	5	[Ex, Gd, TA, Fa, Po]	0	0
CentralAir	object	Categorical	2	[Y, N]	0	0
Electrical	object	Categorical	5	[SBrkr, FuseF, FuseA, FuseP, Mix, nan]	1	0.0684932
1stFlrSF	int64	Numerical	753	[856, 1262, 920, 961, 1145, 796, 1694, 1107, 1...	0	0
2ndFlrSF	int64	Numerical	417	[854, 0, 866, 756, 1053, 566, 983, 752, 1142, ...	0	0
LowQualFinSF	int64	Numerical	24	[0, 360, 513, 234, 528, 572, 144, 392, 371, 390]	0	0
GrLivArea	int64	Numerical	861	[1710, 1262, 1786, 1717, 2198, 1362, 1694, 209...	0	0
BsmtFullBath	int64	Numerical	4	[1, 0, 2, 3]	0	0
BsmtHalfBath	int64	Numerical	3	[0, 1, 2]	0	0
FullBath	int64	Numerical	4	[2, 1, 3, 0]	0	0
HalfBath	int64	Numerical	3	[1, 0, 2]	0	0
BedroomAbvGr	int64	Numerical	8	[3, 4, 1, 2, 0, 5, 6, 8]	0	0
KitchenAbvGr	int64	Numerical	4	[1, 2, 3, 0]	0	0
KitchenQual	object	Categorical	4	[Gd, TA, Ex, Fa]	0	0
TotRmsAbvGrd	int64	Numerical	12	[8, 6, 7, 9, 5, 11, 4, 10, 12, 3]	0	0
Functional	object	Categorical	7	[Typ, Min1, Maj1, Min2, Mod, Maj2, Sev]	0	0
Fireplaces	int64	Numerical	4	[0, 1, 2, 3]	0	0
FireplaceQu	object	Categorical	5	[nan, TA, Gd, Fa, Ex, Po]	690	47.2603
GarageType	object	Categorical	6	[Attchd, Detchd, BuiltIn, CarPort, nan, Basmen...	81	5.54795
GarageYrBlt	float64	Numerical	97	[2003.0, 1976.0, 2001.0, 1998.0, 2000.0, 1993....	81	5.54795
GarageFinish	object	Categorical	3	[RFn, Unf, Fin, nan]	81	5.54795
GarageCars	int64	Numerical	5	[2, 3, 1, 0, 4]	0	0
GarageArea	int64	Numerical	441	[548, 460, 608, 642, 836, 480, 636, 484, 468, ...	0	0
GarageQual	object	Categorical	5	[TA, Fa, Gd, nan, Ex, Po]	81	5.54795
GarageCond	object	Categorical	5	[TA, Fa, nan, Gd, Po, Ex]	81	5.54795
PavedDrive	object	Categorical	3	[Y, N, P]	0	0
WoodDeckSF	int64	Numerical	274	[0, 298, 192, 40, 255, 235, 90, 147, 140, 160]	0	0
OpenPorchSF	int64	Numerical	202	[61, 0, 42, 35, 84, 30, 57, 204, 4, 21]	0	0
EnclosedPorch	int64	Numerical	120	[0, 272, 228, 205, 176, 87, 172, 102, 37, 144]	0	0
3SsnPorch	int64	Numerical	20	[0, 320, 407, 130, 180, 168, 140, 508, 238, 245]	0	0
ScreenPorch	int64	Numerical	76	[0, 176, 198, 291, 252, 99, 184, 168, 130, 142]	0	0
PoolArea	int64	Numerical	8	[0, 512, 648, 576, 555, 480, 519, 738]	0	0
PoolQC	object	Categorical	3	[nan, Ex, Fa, Gd]	1453	99.5205
Fence	object	Categorical	4	[nan, MnPrv, GdWo, GdPrv, MnWw]	1179	80.7534
MiscFeature	object	Categorical	4	[nan, Shed, Gar2, Othr, TenC]	1406	96.3014
MiscVal	int64	Numerical	21	[0, 700, 350, 500, 400, 480, 450, 15500, 1200,...	0	0
MoSold	int64	Numerical	12	[2, 5, 9, 12, 10, 8, 11, 4, 1, 7]	0	0
YrSold	int64	Numerical	5	[2008, 2007, 2006, 2009, 2010]	0	0
SaleType	object	Categorical	9	[WD, New, COD, ConLD, ConLI, CWD, ConLw, Con, ...	0	0
SaleCondition	object	Categorical	6	[Normal, Abnorml, Partial, AdjLand, Alloca, Fa...	0	0
SalePrice	int64	Numerical	663	[208500, 181500, 223500, 140000, 250000, 14300...	0	0

data_type_grp	Categorical	Numerical	Total
cardinality_bin
(0 -- 10]	40	12	52
(10 -- 20]	2	4	6
(100 -- 200]	0	4	4
(20 -- 30]	1	2	3
(200 -- 500]	0	5	5
(500 -- 1000]	0	6	6
(90 -- 100]	0	3	3
1000+	0	2	2
Total	43	38	81

data_type_grp	Categorical	Numerical	Total Non Missing
missing_bin
0	27	35	62
Total Non Missing	27	35	62

data_type_grp	Categorical	Numerical	Total Missing
missing_bin
(0 -- 10]	11	2	13
(10 -- 20]	0	1	1
(40 -- 50]	1	0	1
(80 -- 90]	1	0	1
(90 -- 100]	3	0	3
Total Missing	16	3	19

EDA Code¶

EDA Report for House Price Dataset (Regression)¶

Prepared by - Samarth Agrawal (using Fast ML)¶

EDA for Overall Dataset¶

1. Shape of dataset:¶

2. Size of dataset:¶

3. Dataset info¶

4. Check Cardinality for all the variables in the dataset¶

5. Check Missing Values for all the variables in the dataset¶

EDA For Numerical Variables¶

1. EDA for Variable : Id¶

1. Spread Statistics¶

2. Missing Information¶

3. Histogram and Normality Check¶

4. Variable Distribution with Target Variable¶

5. Scatter Plot between Variable and Target¶

6. Variable Bins with Mean value of Target Variable¶

7. Outlier Analysis¶

8. Box Plots¶

2. EDA for Variable : LotFrontage¶

1. Spread Statistics¶

2. Missing Information¶

3. Histogram and Normality Check¶

4. Variable Distribution with Target Variable¶

5. Scatter Plot between Variable and Target¶

6. Variable Bins with Mean value of Target Variable¶

7. Outlier Analysis¶

8. Box Plots¶

3. EDA for Variable : LotArea¶

1. Spread Statistics¶

2. Missing Information¶

3. Histogram and Normality Check¶

4. Variable Distribution with Target Variable¶

5. Scatter Plot between Variable and Target¶

6. Variable Bins with Mean value of Target Variable¶

7. Outlier Analysis¶

8. Box Plots¶

4. EDA for Variable : YearBuilt¶

1. Spread Statistics¶

2. Missing Information¶

3. Histogram and Normality Check¶

4. Variable Distribution with Target Variable¶

5. Scatter Plot between Variable and Target¶

6. Variable Bins with Mean value of Target Variable¶

7. Outlier Analysis¶

8. Box Plots¶

5. EDA for Variable : YearRemodAdd¶

1. Spread Statistics¶

2. Missing Information¶

3. Histogram and Normality Check¶

4. Variable Distribution with Target Variable¶

5. Scatter Plot between Variable and Target¶

6. Variable Bins with Mean value of Target Variable¶

7. Outlier Analysis¶

8. Box Plots¶

6. EDA for Variable : MasVnrArea¶

1. Spread Statistics¶

2. Missing Information¶

3. Histogram and Normality Check¶

4. Variable Distribution with Target Variable¶

5. Scatter Plot between Variable and Target¶

6. Variable Bins with Mean value of Target Variable¶

7. Outlier Analysis¶

8. Box Plots¶

7. EDA for Variable : BsmtFinSF1¶

1. Spread Statistics¶

2. Missing Information¶

3. Histogram and Normality Check¶

4. Variable Distribution with Target Variable¶

5. Scatter Plot between Variable and Target¶

6. Variable Bins with Mean value of Target Variable¶

7. Outlier Analysis¶

8. Box Plots¶

8. EDA for Variable : BsmtFinSF2¶

1. Spread Statistics¶

2. Missing Information¶

3. Histogram and Normality Check¶

4. Variable Distribution with Target Variable¶

5. Scatter Plot between Variable and Target¶

6. Variable Bins with Mean value of Target Variable¶